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Nowdays, Rasch model analysis is used widely in social research, moreover 
in educational research. In this research, Rasch model is used to determine 
the validation and the reliability of systemic multiple choices question in 
chemistry teaching and learning. There were 30 multiple choices question 
with systemic approach for high school student class XI. In this research, a 
164 student as the subject of the research. The data was collecting during 
March-April 2015. The data was analysis using Winsteps application with 
Rasch model. The reseach shows that 29 question of 30 questions was valid 
through Rasch model analysis. The reliability of systemic multiple choices 
question was 0.93, which is very good chategory. 
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1. INTRODUCTION 

Chemistry teaching and learning is a learning process in understanding the chemistry concepts. As 
well asother subjects, there willbe givenan evaluationto measure theacademicabilitiesof students.There are 
several instrumentsto measure the academic achievement of students, such as multiple choices question, true- 
false question, etc. These kinds of instruments can be categorized in two models, which are paper based 
question and computer based question. 

Multiple choices questions is one formof evaluationthatthe answer can beobtainedby selectingone of 
the answersthathave been provided. This kind of evaluation consists of two parts, whish are a problem (stem) 
and a list of suggested solutions (alternatives) [1]. The basic form of a stemis is a question or an incomplete 
statement. Moreover, the list of the alternatives contains of the best answer (or the correct answer) and a 
number of incorrect answer or inferior alternatives (distractor). 

Multiple choices question can be used to measure various levels of cognitive knowledge, which are 
related to Bloom’s taxonomy of educational objectives (knowledge, comprehension, application, analysis, 
synthesis, and evaluation).The strengths and limitations of this type question are shown in Table 1. 

Usually, multiple choices question has not beenable to reveal the chemistry conceptsin a 
comprehensive manner. As a result, studentsare lessable to thinkconstructivelyandtend to forgetthe 
basicchemistryconceptsthey have learnedpreviously [2], Therefore, it is required a new approach in 
developing a multiple choices questions. 

In this study, the multiple choices questions was developed using systemic approach. Thefore, this 
type of question is namely chemistry systemic multiple choices questions (CSMCQs). The systemic approach 
Is an approach in teaching and learning where multiple components are connected to each other 
byamutualrelationship [3]. The alternatives of SMCQs consists of three concepts in chemistry which are 
connected each other. 
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Table 1. The strengths and the limitations of multiple choices question [4] 


Strengths 

Learning outcomes from simple to complex can be measured. 
Highly structured and clear tasks are provided. 

A broad sample of achievement can be measured. 

Incorrect alternatives provide diagnostic information. 

Scores are more reliable then subjectively scored item (e.g. 
essay). 

Scoring is easy, objective, and reliable. 

Item analysis can reveal how difficult each item was and how 
well it discriminated between the strong and weaker student in 
the class. 

Performance can be compared from class to class and year to 
year. 

Can cover a lot of material very efficiently. 


Limitations 

1. Constructing good items is time consuming. 

2. It is frequentlyo difficult to find plausible distractors. 

3. It is ineffective for measuring some types of prolem 
solving and the ability to organize and express ideas. 

4. Scores can be influenced by reading ability. 

5. Often focus on testing factual information and fails to test 
higher level of cognitive thinking. 

6. Sometimes there is more than on defensible “correct” 
answer. 

7. Does not provide a measure of writing ability. 

8. May encourage guessing. 


As an instrument to measure the academic abilities of student, this kind of evaluation should qualify 
the validity and reliability. Reliability is the degree to which measures are free from error and therefore yield 
consistent results (i.e. the consistency of a measurement procedure) [5]. The instrument is considered reliable 
if the instrument consistently assigns the same score to individuals or objects with equal values. The aim of 
the study was identifying the validity and reliability of CSMCQ though Rasch model analysis. The software 
for analysing the items was WINSTEPS software. 


2. RESEARCH METHOD 

This study is part of research and development aimed at developing an evaluation model based on a 
systemic approach. The kind of evaluation model was systemic multiple choice questions in chemistry 
teaching and learning. The CSMCQs were focussing in two main concepts, i.e. salt hydrolysis and buffer 
solution. The CSMCQs were consists of 30 multiple choices questions. 

Validity determines whether the research trulymeasures that which it was intended to measureorhow 
truthful the research resultsare [6]. The process of validation involves accumulating evidence to providea 
sound scientific basis for the proposed score interpretations [7]. The CSMCQs should qualify the validity, so 
it needs to be tested to the students of class XI. The validity of the evaluation items were tested by 
application Rasch. Items are quantitatively analyzed using WINSTEPS software based on the Raschmodel to 
assess the validity and reliability of items. 


3. RESULTS AND ANALYSIS 

Rasch model with the application of WINSTEPS was used to analyze the datas as well as to test the 
validity and reliability of the instrument. Rasch analysis offers a method of ensuring that key measurement 
assumptions are tested and, where data fit the model, arithmetic operations may be undertaken [8]. The Rasch 
model considers the ability of the students, consists of 164 students, who answered the CSMCQs. 

The items of CSMCQs were valid based on the Rasch model analysis if it complies with the 
following criteria. 

a. The value of Outfit Mean Square (MNSQ) in the range of 0.5 <MNSQ <1.5 

b. The value of Outfit Z-Standard fZSTD) in the range of -2.0 < ZSTD< +2.0 

c. The value of Point Measure Coorelation (Pt Mean Corr) in the range of 0,4 <Pt Measure < 0,85 

Items was valid should meet at least one of the criteria above [9]. 

The validity of an instrument of evaluation is the degree that shows where a test measures what it 
intends to be measured. The value of each item through Rasch model analysis is shown in Table 2. Based on 
these criteria above, it was 1 item, which is number 14, from 30 items which not valid because the value of 
MNSQ, ZSTD and Pt Mean Corr out of the range that is allowed. 

The reliability analysis of CSMCQs performed on 29 valid items, which is shown in Table 3. Based 
on Rasch model analysis using WINSTEPS software, the value of item reliability is 0.93. This value can be 
chategorized as very good chategory [9].Reliability, the consistency of a test or measurement, is frequently 
quantified in the movement sciences literature [10].The realibility of this instrument only conducted in this 
population under certain condition. 

Chemistry is a very complex subject is shown from the research on problem solving 
and misconceptions [11]. Therefore, we need to use new approaches in chemistry teaching and learning so 
that students can understand the chemistry comprehensively. An approach that can be used is a systemic 
approach, where some multiple componentsare connectedto each other byamutualrelationship. It required the 
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evaluation model that supports such approach.The chemistry systemic multiple choices questions (CSMCQs) 
is an instrument to measure the academic abilities of student class XI. The CSMCQs is a set of question 
which has three chemistry concepts connected each other byamutualrelationship in every alternative. The 
CSMCQs consists of 30 items of buffer solution and salt hydrolysis concepts. This material was chosen 
because many students who have misconceptions on this matter [12], 


Table 2. Validity of items CSMCQs trough Rasch Model Analysis 
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Table 3.The reliability of SMCQs 
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Before it can be used, this kind of evaluation should qualify the validity and reliability. The 
instrument validity determines whether the instrumet trulymeasures that which it was intended to measure 
[6], The instrument reliability is the degree to which measures are free from error and therefore yield 
consistent results. Based on the analysis, it is show that 29 itemswas valid and it can be used to 
measureacademic abilities in chemistry teaching and learning with very good reliability. 


4. CONCLUSION 

The result of this study showed that 29 question of 30 questions was valid through Rasch model 
analysis. The reliability of chemistry systemic multiple choices question fCSMCQs) was 0.93, which is very 
good chategory. 
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